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Objective The Ubiquitous Spaced Retrieval-based Memory Advancement and Rehabilitation Training (USMART) program was de- 
veloped by transforming the spaced retrieval-based memory training which consisted of 24 face-to-face sessions into a self-administered 
program with an iPAD app. The objective of this study was to evaluate the feasibility and efficacy of USMART in elderly subjects with 
mild cognitive impairment (MCI). 

Methods Feasibility was evaluated by checking the satisfaction of the participants with a 5-point Likert scale. The efficacy of the pro- 
gram on cognitive functions was evaluated by the Korean version of the Consortium to EstabUsh a Registry for Alzheimer's Disease 
Neuropsychological Assessment Battery before and after USMART. 

Results Among the 10 participants, 7 completed both pre- and post-USMART assessments. The overall satisfaction score was 8.0±1.0 
out of 10. The mean Word List Memory Test (WLMT) scores significantly increased after USMART training after adjusting for age, edu- 
cational levels, baseline Mini-Mental Status Examination scores, and the number of training sessions (pre-USMART, 16.0±4.1; post- 
USMART, 17.9±4.5; p=0.014, RM-ANOVA). The magnitude of the improvements in the WLMT scores significantly correlated with the 
number of training sessions during 4 weeks (r=0.793; p=0.033). 

Conclusion USMART was effective in improving memory and was well tolerated by most participants with MCI, suggesting that it 
may be a convenient and cost-effective alternative for the cognitive rehabilitation of elderly subjects with cognitive impairments. Further 
studies with large numbers of participants are necessary to examine the relationship between the number of training sessions and the 
improvements in memory function. Psychiatry Investig 201 4; 11 :52-58 

Key Words Mild cognitive impairment. Cognitive therapy. Rehabilitation, Memory, Computer user training. Dementia. 



INTRODUCTION 

AJzheimers disease (AD) is one of the most distressing and 
burdensome mental health problems in the aged population, 
and the incidence of dementia is continuously increasing. Al- 
though pharmacological agents, such as acetylcholinesterase 
inhibitors and N-methyl-D-aspartate receptor antagonists, 
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have demonstrated efficacy in the temporal symptomatic con- 
trol of the cognitive decline and daily function of patients with 
AD, their efficacy has not been sufficient to restore premorbid 
function or maintain functional levels in the later stages of pa- 
tients with AD.' Furthermore, no effective medication has 
been found for patients with mild cognitive impairment 
(MCI), which is a high-risk condition for AD. 

Therefore, a multifactorial therapeutic approach that in- 
cludes both pharmacological and non-pharmacological inter- 
ventions is being increasingly advocated in order to optimize 
the cognition, affect, and global functioning of patients with 
MCI or AD.^'' Because memory impairment is the earliest core 
symptom of AD, memory rehabilitation, such as the spaced 
retrieval training (SRT), is a critical part of the non-pharmaco- 
logical treatments for AD.'* SRT is a method of learning and 



52 Copyright O 2014 Korean Neiiropsychiatric Association 



JW Han et al. 



retaining target information by recalling that information over 
increasingly longer intervals.^ It was originally developed for 
improving memory in cognitively intact persons and was 
based on the expanding rehearsal technique.'''^ The modifica- 
tion was designed to reduce the learner s effort during training, 
reduce the number of errors by matching the difficulty of the 
task to the patients ongoing performance (shaping paradigm), 
and make the training sessions social and enjoyable/ SRT has 
been reported to improve learning and target information re- 
tention and to ameliorate behavioral problems in various types 
of dementia, including AD7 

In our previous work, we developed a Spaced Retrieval- 
based Memory Advancement and Rehabilitation Training 
(SMART) program, which consisted of 24 face-to-face ses- 
sions." SMART was effective in improving the memory reten- 
tion spans of patients with very mild to mild AD. Further- 
more, the expanded retention span induced by SMART was 
maintained for different sets of target information, indicating 
that the efficacy of SMART can be generalized. 

However, SMART has several limitations in its current form 
that prevent it from being provided to as many individuals 
who may benefit from it as possible. First, more than an hour 
of face-to-face contact with a trained therapist is required for 
each session, and, thus, it has high costs. Furthermore, trained 
therapists are not available in many areas. Second, it is not easy 
to maintain compliance because both patients and caregivers 
need to regularly allow for about 3 hours per visit and 3 visits a 
week. Third, it is difficult to increase the intensity of the train- 
ing, such as the length of the retention span and the number of 
training sessions per week, due to the previously described 
limitations. In order to overcome these limitations of SMART, 
we developed a Ubiquitous SMART (U- SMART) program by 
transforming the current SMART program into a self-admin- 
istered program with an iPAD app, and we examined its feasi- 
bility and efficacy in elderly individuals with MCI. 

METHODS 
Subjects 

Ten elderly individuals with MCI participated in the present 
study; 4 were enrolled from the participants of the Korean 
Longitudinal Study on Cognitive Aging and Dementia 
(KLOSCAD), and 6 were enrolled from the visitors to the De- 
mentia Clinic of the Seoul National Uruversity Bundang Hos- 
pital. The KLOSCAD is a population-based prospective elderly 
cohort study on cognitive aging and dementia that was 
launched in 2009. 

Each subject was evaluated with the Korean version of the 
Consortium to Establish a Registry for Alzheimer's Disease 
(CERAD-K)" by a research geropsychiatrist with expertise in 



dementia research. MCI was diagnosed according to the Re- 
vised Diagnostic Criteria for MCI proposed by the Interna- 
tional Working Group on MCI.'^ The subjects who had Axis I 
disorders that are listed in the DSM-IV"" or serious medical or 
neurological disorders that could affect their cognitive func- 
tion were excluded. 

This study's protocol was approved by the Institutional Re- 
view Board of Seoul National University Bundang Hospital 
(SNUBH). All subjects were fully informed of the study proto- 
col AU subjects provided written informed consents that were 
signed by the subjects. 

Development of USMART 

USMART was based on a client-server model in which all 
data were stored in a central server, while the iPAD mobile ap- 
plication was designed and implemented in order to provide 
patients with a self-administered SRT program. A web portal 
was developed to provide service providers, such as therapists, 
with accessibility and management capabilities of the data 
(Figure 1). The client program and server system communicat- 
ed over the network with a Hypertext Transfer Protocol Secure 
(HTTPS) in order to ensure secure communication. 

As shown in Figure 1, the central server consisted of the fol- 
lowing different layers of servers in order to support the effi- 
cient development of the mobile and web applications: the 
open application programming interface (API), which acted 
as a gateway between the mobile application and central stor- 
age; the controller layer, which provided a web service for pro- 
viders; the data access layer, which provided queries to access 
the underlying database; and the database server in the persis- 
tence layer All server programs were implemented on a Java 
platform. 

The iPAD mobile application, which was a SRT program for 
patients, was designed to be elderly- friendly and easy to use by 
using high-chroma colors, minimizing user options, providing 
step-by-step voice-guided directions, and providing various 
types of input modes, such as voice recognition, keyboard typ- 
ing, and writing recognition. With the application, patients are 
allowed to have a training session whenever they want, regard- 
less of their location and time. Their training records are then 
automatically sent to and stored in the central database through 
the open API. The current mobile application was tuned to 
operate with a set of 88 words. However, it also provided a 
functionality that allowed patients to train with their own user- 
defined words. One hundred and twenty functional games 
were embedded and provided to patients as part of the inter- 
retrieval activities. 

The development of a web portal for providers enabled pro- 
viders to register a new patient, access the patient's training histo- 
ry, check the patient's progress, and evaluate the training results. 
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Administration of USMART 

Two occupational therapists trained each participant on 
how to use the USMART application and iPAD. Then, each 
participant was asked to use the USMART 3 or more times 
per week for 4 weeks at home. During the 4 weeks, the partici- 
pants were allowed to ask the 2 research occupational thera- 
pists for directions of the USMART anytime by phone or at a 
hospital visit. 

In each USMART session, the subject was asked to remem- 
ber a given set of words that appeared in the screen and then 
recall them immediately. The subject could recall by voice rec- 
ognition or direct keyboard typing. If the subject succeeded in 
recalling the words, he/ she was asked to recall them repeatedly 
over systematically expanded intervals (0.75, 1.5, 3, 6, 12, and 
24 min).'' Following a recall failure, the interval was reduced 
to that of the previous trial. The set of words consisted of 1 to 5 
unrelated high-imagery words (e.g., bus, grape, duck, scissors, 
etc.). All targets were disyllabic words in the Korean language, 
and different words were used in all sessions.'' If the subject 
succeeded in recalling the given set of words for 24 min in 3 
consecutive sessions, the number of words in the set was se- 
quentially increased in the subsequent session by an auto- 
mated algorithm in the USMART. The starting number of 
words in the first session was 1. During each interval, the 
subjects were asked to perform inter-retrieval activities in or- 
der to prevent them from rehearsing the given set of words. 
The inter-retrieval activities consisted of a quiz that could 
improve attention/concentration, visual memory, orienta- 
tion, and categorization. All procedures during the USMART 
training were guided by automatic verbal messages in the 



Figure 1. System architecture of the 
Ubiquitous Spaced Retrieval-based 
IVIemory Advancement and Rehabilita- 
tion Training (USMART). 

USMART apparatus (i.e., the iPAD). 

Assessments of feasibility and efficacy 

We evaluated the feasibility of USMART by assessing the 
satisfaction and compliance of the participants. We measured 
the level of satisfaction with USMART with a 17-item satisfac- 
tion questionnaire (4 items on the content of the program, 2 
items on accuracy, 2 items on the layout of the program, 2 
items on the ease of use, 2 items on timehness, 3 items on the 
program speed, and 2 items on overall satisfaction). Each item 
was scored with a 5-point Likert scale from 1 (not at all) to 5 
(very much). 

We evaluated compliance by assessing the number of self- 
training sessions that were performed for 4 weeks. 

In order to evaluate the efficacy of USMART on cognitive 
functions, we administered the Korean version of the CERAD 
Neuropsychological Assessment Battery (CERAD-K-N), the 
Korean version of the Frontal Assessment Battery,'^ and the 
Digit Span Test'* before and after subjects performed the 
USMART program. The CERAD-K-N consists of 9 neuropsy- 
chological tests, including the Categorical Fluency test, the 
Modified Boston Naming Test, the Mini-Mental State Exami- 
nation (MMSE), the Word List Memory Test (WLMT), the 
Constructional Praxis Test, the Word List Recall Test, the 
Word List Recognition Test, the Constructional Recall Test, 
and the Trail Making Test A. These pre- and post-treatment 
assessments were performed within 2 weeks before or after the 
performance of the 4-weeks' USMART program. 
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Statistical analysis 

The Mann-Whitney U-test was used to compare the age, 
educational levels, and baseline MMSE scores between the 
subjects who completed the U-SMART program and those 
who dropped out. 

Repeated measures analysis of variance (RM-ANOVA) was 
used to compare the pre- and post-USMART neuropsycho- 
logical performances after adjusting for age, educational levels, 
pre-USMART MMSE scores, and the number of training ses- 
sions that were performed for 4 weeks. For any of the neuro- 
psychological tests that showed improvement by the training, 
a correlation analysis using Spearman's rank correlation coef- 
ficient was performed to examine the factors that determined 
the degree of improvement. For our statistical evaluation of 
the main efiects, P values less than 0.05 were considered statis- 
tically significant. All statistical analyses were performed with 
the PASW Statistics 18, Release Version 18.0.0 (O SPSS, Inc., 
Chicago, IL, USA). 

RESULTS 

The demographic characteristics of the 10 participants (1 
man, 9 women) are summarized in Table 1. Among them, 7 
women (4 amnestic MCI single-domain type, 2 amnestic MCI 
multiple-domain type, 1 non-amnestic MCI single-domain 
type) completed both the pre- and post-USMART assess- 
ments. The number of training sessions that were performed 
by these 7 participants for 4 weeks varied widely from 1 to 30. 
Among them, 1 subject (ID 7) had been taking donepezU (10 
mg) for 6 months at the beginning of this study. Three subjects 
dropped out for the following reasons: 1 subject (ID 8) had 
difficulty learning how to use the iPAD, 1 (ID 9) felt the inter- 
retrieval activities were childish and 1 (ID 10) developed an 
acute physical illness. However, their age (p=0.085, Mann- 



Whitney U-test), education (p=0.816, Mann- Whitney U-test), 
and baseline MMSE scores (p=0.728, Mann-Whitney U-test) 
were comparable to those of the 7 subjects who completed the 
assessments. 

The overall satisfaction score was 8.0±1.0 out of 10 points. 
AU participants scored 4 or 5 in the overall satisfaction items 
and answered that they were willing to purchase the USM ART 
program. Five responders (71.4%) answered that the USMART 
was helpful in improving their memory. The points for 15 out 
of the 17 items were 3 or more. Mean scores of the content, ac- 
curacy, layout, ease of use, timeliness, and program speed were 
12.7±3.1 out of 20 points, 7.4±1.5 out of 10 points, 7.3±1.5 out 
of 10 points, 7.1±1.7 out of 10 points, 6.6±2.0 out of 10 points, 
and 9.4±2.9 out of 15 points, respectively. 

As shown in Table 2, the mean WLMT score significantly 
increased after the USMART training (pre-USMART, 16.0 ± 
4.1; post-USMART, 17.9±4.5). After adjusting for age, educa- 
tional level, baseline MMSE score, and the number of training 
sessions, this change was statistically significant (F(l, 2) 
=70.327, p=0.014). Figure 2 shows the changes in the WLMT 
scores after USMART in each participant. Furthermore, the 
magnitudes of the improvements in the WLMT scores signifi- 
cantly correlated with the number of training sessions per- 
formed during the 4 weeks (r=0.793, p=0.033) (Figure 3). 

DISCUSSION 

We found that performing the USMART program for 4 
weeks was applicable to a considerable proportion of the el- 
derly patients with MCI, and it was helpful in improving word 
list learning in proportion to the number of training sessions. 

We were concerned whether elderly people with cognitive 
impairments were able to use the USMART program by them- 
selves. However, most participants seemed to adapt readily to 



Table 1 . The characteristics of the participants 



ID 


Age (y) 


Gender 


Education (y) 


Diagnosis 


CDR 


Session* 


1 


61 


Female 


9 


MCI-AM 


0.5 


30 (44) 


2 


67 


Female 


6 


MCI-AS 


0.5 


7(14) 


3 


60 


Female 


6 


MCI-AS 


0.5 


13(19) 


4 


74 


Female 


6 


MCI-AS 


0.5 


21 (28) 


5 


73 


Female 


16 


MCI-NS 


0.5 


12(12) 


6 


73 


Female 


12 


MCI-AS 


0.5 


7(12) 


7 


80 


Female 


9 


MCI-AM 


0.5 


1(3) 


8 


79 


Female 


5 


MCI-AM 


0 


0 


9 


79 


Female 


10 


MCI-AM 


0.5 


0 


10 


75 


Male 


16 


MCI-AS 


0 


0 



*totaI number of training sessions during 4 weeks. MCI: mUd cognitive impairment, AM: amnestic multiple-domain type, AS: amnestic sin- 
gle-domain type, NS: non-amnestic single-domain type, CDR: CUnical Dementia Rating 



www.psychiatryinvestigation. org 55 



Ubiquitous Spaced Retrieval-Based IVlemory Training 



Table 2. The effect of Ubiquitous Spaced Retrieval-based Memory Advancement and Rehabilitation Training (USMART) on the cognitive 
functions of elderly individuals with mild cognitive impairment (MCI) 



Pre-USMART 



Post-USMART 



Mini-Mental State Examination 


26.7±L9 


25.2±L8 


0.068 




Categorical Fluency Test 


12.0±3.2 


1L3±2.4 


0.343 


0.401 


Boston Naming Test 


1L4±2.3 


1L9±0.9 


0.671 


0.756 


Word List Memory Test 


16.0±4.1 


17.9±4.5 


0.246 


0.014 


Word List Recall Test 


4.4±2.2 


5.4±3.6 


0.345 


0.279 


Word List Recognition Test 


8.3±L8 


8.6±L9 


0.891 


0.847 


Construction Praxis Test 


10.1±L5 


10.4±0.8 


0.414 


0.191 


Construction Recall Test 


4.9±3.0 


5.6±3.0 


0.496 


0.598 


Trail Making Test-A 


5L0±15.4 


52.6±13.3 


0.498 


0.836 


Trail Making Test-B 


204.1 ±76.0 


188.0±74.7 


0.128 


0.063 



*Wilcoxon signed-rank test, ^repeated Measured Analysis of Variance (RM-ANOVA), adjusted for age, educational level, MMSE score before 
the USMART, and the number of training sessions for 4 weeks 
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Figure 2. The changes in the Word List Memory Test scores after 
USMART in each participant. MCl-AM: mild cognitive impairment- 
amnestic multiple-domain type, AS: amnestic single-domain type, 
NS: non-amnestic single-domain type. 

the USMART program after several introductions and did not 
need to have a caregiver or therapist present during training, 
indicating that the USMART program may not be overwhelm- 
ing or emotionally taxing to elderly patients with MCI. How- 
ever, 1 participant dropped out because he failed to learn how 
to use the iPAD, and 5 of the 7 participants who completed the 
current study needed 2 or more teaching sessions to learn how 
to use the iPAD properly. In addition, the participants who had 
difficulty learning how to use the iPAD had a lower number of 
training sessions. If smart devices like the iPAD will become 
more popular and familiar to elderly individuals in the future, 
the USMART program will become more widely and easily 
applicable to elderly patients with cognitive impairments. The 
7 participants showed high levels of overall satisfaction. How- 
ever, their satisfaction with each aspect of the program and its 
timeliness and speed in particular was somewhat lower than 
their overall satisfaction. In addition, some participants felt 
that the inter-retrieval activities were too easy and childish. 



10 



< 



r=0.793, p=0.033* 



0 5 10 15 20 25 30 

Number of training sessions during 4 weeks 

Figure 3. Correlation between the number of training sessions 
and the magnitude of the improvements in the Word List Memory 
Test (WLMT) scores. *Spearman's rank correlation coefficient. 

The inter-retrieval activities were tailored to each subject in or- 
der to enhance the compliance with the USMART program. 

We found that USMART dose-dependently improved the 
memory of elderly individuals with MCI. These data add to 
the growing empirical evidence on the benefits of spaced re- 
trieval on the memory function of patients with MCIs." In this 
study, the efficacy of the USMART program was confined to 
memory aspects that were measured by the WLMT. Our ob- 
servation was consistent with earlier studies that reported that 
objective measures of memory reveal statistically significant 
improvements at the end of training when compared to objec- 
tive measures of cognition other than memory.^ The effective- 
ness of spaced retrieval as a method of facilitating memory 
function has been variously attributed to the effects of repeti- 
tion priming, operant conditioning, classical conditioning, and 
errorless learning, but there is a general consensus that at 
least some of the effects of spaced retrieval are attributable to 
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its reliance on automatic processing.^"^^ Different cognitive 
processes are subjected to different rates and degrees of decline 
in both patients with dementia and MCI; controlled cognitive 
processing is more substantially affected than automatic pro- 
cessing.^' The USMART program may be helpful for improv- 
ing the storage of target information in the everyday lives of 
patients with MCI. 

Furthermore, the magnitude of memory improvement was 
correlated with the number of training sessions in the present 
study, which further supported the findings that the USMART 
program maybe effective in improving memory. Some studies 
have suggested that participants become more proficient at the 
SRT task with an increasing number of sessions, which is re- 
flected in the longer retention intervals in the later sessions 
compared to the early sessions and in the number of failures 
that decreased for each participant across the sessions.^''"^'' 
These explanations provide additional evidence that, over time, 
participants benefit from prior training sessions. However, the 
direct effect of the increased training sessions on cognitive im- 
provement has not been fully investigated yet. One previous 
study that included 60 training sessions^^ did not show any su- 
periority compared to the other studies involving an inferior 
number of sessions, and others have suggested that those pro- 
grams that apply fewer sessions (between 6 and 20) seem to be 
more cost-effective for future research and clinical purposes.^ 
The number of training sessions for the 4 weeks in our study 
had a mean value of 13 (s.d., 9.7). Therefore, the duration of 
our study might not be categorized as long term. Further stud- 
ies that include a large number of participants are necessary to 
examine the relationship between the number of training ses- 
sions and the improvements in memory function. 

According to the current study, the memory-enhancing ef- 
fects of the USMART program might be generalized because 
the performance of the word list learning test, which employed 
a set of words that were totally different from those employed 
in the USMART program, was significantly improved after the 
USMART program. In our previous study on the SMART pro- 
gram, however, SMART was effective in expanding the reten- 
tion span of words that were very different from the words 
that were employed in the SMART training but not in improv- 
ing the performance of the word list learning test." This dis- 
crepancy between the effects of USMART and SMART on the 
word list learning test may be partly attributed to differences in 
the severity of the cognitive impairments in the subjects in the 
2 studies. In the previous study on SMART, all subjects were 
AD patients, whereas all were MCI patients in the present 
study. Because more cogrutive effort and explicit memory may 
be required for conventional neuropsychological tests like the 
word list learning test compared to SMART or USMART,*^''^ 
the effect size of SMART or USMART on the performance of 



the WLMT may be smaller in AD patients who have more se- 
vere impairments in explicit memory than in individuals with 
MCI. 

Although the current study showed that USMART might 
be feasible and useful in elderly individuals with MCI, several 
limitations should be noted. First, the sample size was small, 
the treatment period was short, and the study design was an 
open-label single arm. Second, the type of MCI was not uni- 
form. The feasibility and efficacy of the USMART may be dif- 
ferent depending on the type of MCI, i.e., amnestic versus 
non-amnestic. Third, long-term efficacy was not evaluated in 
the present study. Because MCI is a degenerative condition in 
at least a certain proportion of individuals,^* ™ it is crucial to 
assess the longitudinal course of the efficacy data. A delay of 
the conversion toward dementia could eventually be consid- 
ered the ultimate efficacy outcome. Fourth, medication effects 
were not totally excluded in this study. One subject had been 
taking donepezU, and 1 subject had been taking escitalopram. 
However, the changes in the WLMT scores after USMART 
did not differ in the subjects taking these drugs. The WLMT 
score of the subject who was taking donepezil decreased 1 
point and ffiat of the subject who was taking escitalopram in- 
creased 1 point after USMART. 

In conclusion, USMART was effective in improving memo- 
ry, and it was well tolerated by most participants with MCI, 
suggesting that it may be a convenient and cost-effective alter- 
native for cognitive rehabilitation in elderly patients with cog- 
nitive impairments. Information technology can play a key 
role in making conventional cognitive training more accessible 
and convenient to elderly patients with cognitive impairments. 
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